Adaptable input/output device

ABSTRACT

The present invention relates to an adaptable input/output device. One embodiment of a hardware device for facilitating an interaction between a computing system and a user includes: an interaction surface for supporting the interaction, a single actuator capable of driving a first region of the interaction surface, and a first selective clamping mechanism capable of restricting movement of one or more second regions of the interaction surface that partly intersect the first region, wherein a displacement of one or more desired portions of the interaction surface is dynamically controllable.

FIELD OF THE INVENTION

The present invention relates generally to devices technology forinteracting with computer systems (or other electronic devices withcomputational abilities such as electronic instruments, microprocessorcontrolled displays, or the like), and relates more particularly toinput/output devices used with computer systems.

BACKGROUND OF THE DISCLOSURE

Conventional input/output (I/O) devices such as keyboards and mice donot adapt to an end user's needs or working habits in the sense that theI/O devices typically cannot adjust their physical shape in response tothe user's interactive context. For example, while the functionalityassociated with particular keys on a conventional computer keyboard canbe reassigned by software to a variety of different functions, theconventional keyboard remains a keyboard: it is not designed or enabledto dynamically change shape and transform (e.g., into a joystick) inresponse to the current usage context

Moreover, conventional I/O devices tend to occupy a significant amountof the user's available working space; thus, a keyboard may compete andconflict with a display over limited surface area. The space conflict isespecially problematic when dealing with portable computing devices(e.g., laptop computers, personal digital assistants, and the like).Furthermore, while the various regions of a touch-enabled display screencan be dynamically reassigned to different functions, the physical shapeof the display screen is conventionally fixed and remains asubstantially flat surface. This results, among other limitations, inlittle or no meaningful tactile feedback for the user, and is less thanoptimal for many interactive applications.

Existing or proposed displays that can change shape out-of-plane (e.g.,Braille displays) generally rely on individual actuators to control theout-of-plane position of individual display elements. This approachentails a large number of actuators, has performance limitations, andcan be complex, unreliable, and costly.

SUMMARY OF THE INVENTION

The present invention relates to an adaptable input/output device. Oneembodiment of a hardware device for facilitating an interaction betweena computing system and a user includes: an interaction surface forsupporting the interaction, a single actuator capable of driving a firstregion of the interaction surface, and a first selective clampingmechanism capable of restricting movement of one or more second regionsof the interaction surface that partly intersect the first region,wherein a displacement of one or more desired portions of theinteraction surface is dynamically controllable.

In another embodiment, a hardware device for facilitating an interactionbetween a computing system and a user includes: an adaptable surface forsupporting the interaction, the adaptable surface being dynamicallydeformable under a control of the computing system so as to produce adeformation, a controllable clamping mechanism capable of varying aresistance of the deformation to movement in response to a pressureapplied by the user, and circuitry for transmitting an input signal tothe computing system responsive to the pressure.

In another embodiment, a hardware display device for facilitating aninteraction between a computing system and a user includes an adaptablesurface for supporting the interaction, the adaptable surface beingdynamically deformable under a control of the computing system so as toproduce a deformation that substantially mimics an appearance associatedwith an input device, wherein the deformation is further deformableunder a user-applied pressure to assume a shape that substantiallymimics the input device when pressed, and circuitry for transmitting aninput signal to the computing system in response to the user-appliedpressure.

BRIEF DESCRIPTION OF THE DRAWINGS

The teachings of the present invention can be readily understood byconsidering the following detailed description in conjunction with theaccompanying drawings, in which:

FIG. 1 is a schematic diagram illustrating one embodiment of a hardwaredevice for facilitating interactions between a user and a computingsystem, according to the present invention;

FIG. 2 depicts an exemplary set of grid elements that is elevated tosimulate a button or key;

FIG. 3 illustrates an exemplary set of grid elements that is configuredto simulate a slider;

FIG. 4 illustrates an exemplary user interface that can be displayed onthe display layer;

FIG. 5 is a schematic diagram illustrating a top view of one embodimentof the polymorphic layer, according to the present invention

FIG. 6 is a schematic diagram illustrating a cross-sectional view of oneembodiment of the polymorphic layer illustrated in FIG. 1;

FIGS. 7A-7C illustrate various embodiments of a trap door-like selectormechanism;

FIG. 7D illustrates an alternative selector mechanism;

FIG. 8 illustrates a close-up view of the selector mechanism illustratedin FIG. 6;

FIGS. 9A-9B illustrate another alternative embodiment of the selectormechanism illustrated in FIG. 6;

FIG. 10 is a schematic diagram illustrating a cross-sectional view ofanother embodiment of the polymorphic layer illustrated in FIG. 1;

FIGS. 11A-11C are schematic diagrams illustrating top views of variousexemplary embodiments of clamping mechanisms employing joints, accordingto the present invention;

FIGS. 12A-12C are schematic diagrams illustrating various embodiments oflocking mechanisms that may be employed to selectively lock the flexiblejoints illustrated in FIGS. 11A-11C;

FIG. 13 illustrates one embodiment of a piece of flexible or compliantfabric;

FIG. 14 illustrates one embodiment, of an accordion folded fabric thatmay be deployed above any of the clamping mechanisms discussed herein;

FIG. 15 illustrates another embodiment of a multi-layered fabric thatmay be deployed above any of the clamping mechanisms discussed herein;

FIG. 16 illustrates another embodiment of a multi-layered fabric thatmay be deployed above any of the clamping mechanisms discussed herein;

FIG. 17 is a flow diagram illustrating one embodiment of a method forinteracting with a user of a computing system, according to the presentinvention;

FIG. 18 is a flow diagram illustrating one embodiment of a method foradjusting the polymorphic layer of the hardware device;

FIG. 19 illustrates a cellular telephone having an integrated hardwaredevice such as the hardware device illustrated in FIG. 1;

FIG. 20 is a high level block diagram of the present inventionimplemented using a general purpose computing device;

FIG. 21 is an exploded view illustrating a portion of adaptableinput/output device having a curved surface;

FIG. 22 is a schematic diagram illustrating a grid element of apolymorphic surface in which a display tile is integrated; and

FIG. 23 is a schematic diagram illustrating one embodiment of a gridelement array incorporating selector mechanisms such as thoseillustrated in FIG. 7C.

To facilitate understanding, identical reference numerals have beenused, where possible, to designate identical elements that are common tothe figures.

DETAILED DESCRIPTION

The present invention relates to an adaptable input/output (I/O) device.Embodiments of the present invention replace the conventional keyboardand mouse combination (or other X-Y I/O device such as a track pad,touch screen, or track ball) with a single I/O device (e.g., a flatdisplay) that can dynamically adapt its appearance and structure inresponse to changing use contexts. Adaptation of the adaptable I/Odevice may be based on triggers including the user's hand position, theuser's gestures, active applications, active input fields, and displayconfiguration, among others.

FIG. 1 is a schematic diagram illustrating one embodiment of a hardwaredevice 100 for facilitating interactions between a user and a computingsystem, according to the present invention. The device 100 may operateas a standalone computing device (i.e., the device 100 may incorporateor be incorporated into the computing system), or the device 100 may becoupled to a separate computing system 108. The computing system may be,for example, a personal computer, a tablet computer, a cellulartelephone, a smart phone, a gaming console, a handheld gaming device, aset top box, an Internet ready television, a computer-controlled machinesuch as a robot or vehicle, or the like. The device 100 may also be anadaptable display (not facilitating user input itself), wherecomputation enables the device 100 to achieve desired three-dimensionalsurface configurations as directed by the computing system). The device100 illustrated in FIG. 1 is generalized in order to illustrate thebasic components of the present invention.

As illustrated, one embodiment of the device 100 generally comprisesthree layers: a polymorphic layer 102, a display layer 104, and anobservation layer 106. The three layers cooperate in order to interactwith a user in a manner that adapts to changing use contexts.

The polymorphic layer 102 comprises a planar or non-planar interactionsurface that provides a tactile interface through which the user canprovide inputs to the device 100 and, consequently, the computingsystem. Additionally, the device 100 may use the polymorphic layer 102to provide tactile output to the user. To this end, the surface of thepolymorphic layer 102 is adaptable; that is, the surface of thepolymorphic layer 102 is capable of dynamically changing its shape andtexture, under the control of the computing system. In one embodiment,the polymorphic layer 102 additionally includes a vibration mechanism.Additionally, the polymorphic layer 102 is preferably transparent, suchthat the display layer 104, which is positioned beneath the polymorphiclayer 102, is viewable through the polymorphic layer 102. Alternatively,in some embodiments, display elements may be positioned over thepolymorphic layer 102, as discussed in greater detail below.

In one embodiment, the polymorphic layer 102 is formed of a plastic(e.g., acrylic) three-dimensional transparent micro grid comprising aplurality of grid elements and at least one actuator. The arrangement ofthe grid elements allows the polymorphic layer 102 to simulate aplurality of different input devices.

In another embodiment, the polymorphic layer 102 comprises a flexible orcompliant fabric layer whose shape and texture are locally variableusing a plurality of pivoting, lockable joints between rigid elementsdisposed beneath the fabric layer. In another embodiment, thepolymorphic layer 102 comprises a plurality of directionally flexible orcompliant fabric layers that clamp together. In yet another embodiment,the polymorphic layer 102 comprises a flexible or compliant fabric layerwhose shape, texture, and stiffness are locally variable using aplurality of interwoven strips disposed beneath the fabric layer. Asused herein, the term “fabric” may refer to a textile material, or mayalternatively refer to a material that incorporates plastics, filaments,metals, and/or mixed materials engineered with appropriate properties.

Specific, exemplary structural embodiments of the polymorphic layer 102are described in further detail below with respect to FIGS. 5-16.

To illustrate an example of the type of input devices that can besimulated by the polymorphic layer 102, FIG. 2 depicts an exemplary set200 of grid elements that is elevated to simulate a button or key(horizontal lines are shown in FIG. 2 only to help the reader moreeasily visualize the relative vertical displacement, and are notintended as a depiction of structure). As the user depresses the buttonor key, the set 200 of grid elements gradually lowers with resistance tothe applied pressure to simulate, for example, the response of a buttonon a conventional mouse or a key on a conventional keyboard. In oneembodiment, the resistance of the button or key (i.e., the set 200 ofgrid elements) to such a user-applied pressure is variable (e.g., usinga controllable clamping mechanism, as described in greater detailbelow). Thus, the set 200 of grid elements does not just simulate abutton or key in terms of shape, but also in terms of feel and response.

In further embodiments, the displacement and/or resistance of the set200 of grid elements can be further controlled using a controllableratcheting mechanism.

It will be appreciated that the set 200 of grid elements is provided asan example only; the grid elements may be actuated to take any form, andthe form will vary depending upon the currently active applications anduse contexts. For example, FIG. 3 illustrates an exemplary set 300 ofgrid elements that is configured to simulate a slider. A slider (orscroll bar) is a familiar user input instrumentality, allowing the userto increase or decrease an input parameter value for the computingsystem, by sliding or dragging the corresponding indicator on theslider. In one embodiment of the present invention, a first grid element300 ₁ gives tactile feedback when the upper limit is reached (i.e., whenthe user's finger cannot slide the slider to a higher value). Similarly,a second grid element 300 _(n) gives tactile feedback when the lowerlimit is reached (i.e., when the user's finger cannot slide the sliderto a lower value). The upper and lower limits of the slider are enforcedby clamping the first and second grid elements 300 ₁ and 300 _(n), whichare located at or near the upper and lower edges of the slider, at anelevated position. The user cannot depress the first and second gridelements 300 ₁ and 300 _(n) when they are clamped in this way. As such,the user can know that a limit has been reached without having to lookat the slider. The remaining grid elements in the set 300 of gridelements may be depressed by the user to indicate a current applicablevalue in the range of the slider; thus, the user can change the inputvalue by dragging his or her finger across the top surface of theremaining grid elements. Additionally, a user interface or graphicalelement on the display surface will also indicate the value range(including the edges or limits) of the slider.

In other embodiments still, a set of grid elements may be dynamicallyconfigured to simulate the look and feel of a different kind of userinput device, including at least one of: an alphanumeric keyboard, atelephone-style keypad, a numeric keypad, a media player controller, ajoystick, a video game controller, a television-style remote controller,or a vehicle or robot controller. In another embodiment, the polymorphiclayer 102 is configured to function as a button for an interactivecommand for a window displayed on the display surface 104.

The interactive command may comprise, for example “close the window,”“minimize the window,” “scroll the contents of the window,” or the like.In further embodiments still, the polymorphic layer 102 is configured asa topographic terrain map.

Referring back to FIG. 1, the display layer 104 is positioned beneaththe polymorphic layer 102 and is viewable through the polymorphic layer102. The display layer 104 preferably comprises a flat panel display andprovides visual output to the user. To this end, the display is capableof changing its appearance under the control of the computing system. Inone embodiment, the display layer 104 may serve as the primary displayfor the computing system. In another embodiment, the display layer 104provides two main functions. First, the display layer 104 extends aprimary display to which the device 100 may be coupled (e.g., thedisplay of computing system 108). This allows the display area to bedynamically adjusted/controlled. Second, the display layer 104 outputs(displays) graphical interfaces or user interfaces associated with thecurrent configuration of the polymorphic layer 102 (e.g., a displaykeyboard, a touchpad, or the like). The display layer 104 may comprise,for example, a plasma, liquid crystal, or light emitting diode display.

FIG. 4, for example, illustrates an exemplary user interfaceinstrumentality (or “widget”) 400 that can be displayed on the displaylayer 104. The exemplary user interface 400 includes a keyboard 402 andtwo track pads 404 a and 404 b (hereinafter collectively referred to as“track pads 404”). The exemplary user interface 400 may be useful, forexample, in conjunction with a computing device that uses multipledisplays (e.g., a dual monitor display). For example, one track padcould be provided for each display to enable immediate navigation. In adual monitor display, for instance, use of the exemplary user interface400 could avoid an approximately five thousand pixel initial positioningmovement (if the user must drag the mouse from a first end of a monitorto an opposing second end of the monitor).

The shape and texture of the polymorphic layer 102 is adaptable, underthe control of the computing system, to provide the appropriate tactilesurface for each of the keys in the keyboard 402 and each of the trackpads 404. For example, as discussed above with respect to FIG. 2, a setof grid elements in the location of each displayed key may be elevatedto simulate the look and feel of a key that may be pressed by the user.The user interface 400 may be programmable with respect to any of itsprimary parameters (e.g., language, layout, design, ergonomic factors,colors, sensitivity, and the like). It will be appreciated that the userinterface 400 is provided as an example only; the user interfacesdisplayed on the display layer 104 may take any form and will varydynamically depending upon the currently active applications and usecontexts. For example, the user interface could also be configured as amedia player controller or any of the other input devices describedabove.

Referring back to FIG. 1, the observation layer 106 comprises one ormore sensors that observe and track the user's actions with respect tothe computing system. In one embodiment, the sensors are arranged in agrid. The sensors may include, for example, cameras (e.g., infrared orvisible spectrum micro cameras), capacitive sensors, pressure sensors,or the like. In some embodiments, the observation layer can be anelement of a multi-touch sensing system based on Frustrated TotalInternal Reflection (“FTIR,” a touch-sensing technology known to skilledpractitioners in the field). The sensors preferably also enablemeasurement of the proximity of objects/actions at a distance from thesurface—such as stereo and/or multiple cameras that enable the computingsystem to measure the distance of an object or body part (e.g., finger)from the surface of the device—as discussed in further detail below.Thus, the sensors—in conjunction with the computing system (whichanalyzes the sensor data)—can preferably observe and track a pluralityof actions and movements, including the user's hand position, gestures,and gaze (suitable algorithms for recognizing gestures and the like arefamiliar to those of skill in the art). In addition, the computingsystem can also consider the applications that are currently active onthe computing system, the input field that is currently active on thecomputing system, and the current display configuration (e.g., singleversus multiple monitor display), along with any of the recognizedactions or gestures mentioned above, in order to responsively adapt thepolymorphic layer 102 and the display layer 104. In one embodiment, theobservation layer 106 includes a network interface that allows theobservation layer 106 to exchange data directly with the computingsystem.

In one embodiment, and as depicted in FIG. 1, the observation layer 106is positioned beneath the display layer 104. Thus, the display layer 104is effectively “sandwiched” between the observation layer 106 and thepolymorphic layer 102. Particularly in such embodiments, the polymorphiclayer 102 is preferably made of a substantially transparent material, topermit a substantially unobstructed view of the display layer 104.However, alternative embodiments are also possible in accordance withthe present invention (e.g., the display layer 104 may be implemented ontop of the polymorphic layer 102 and/or integrated with the polymorphiclayer 102). For instance, the display layer 104 may comprise a pluralityof individual miniature display tiles (e.g., liquid crystal display(LCD) tiles) fixed atop a polymorphic surface and able to move with thepolymorphic surface; the display layer 104 may then be projected fromabove onto the polymorphic surface. FIG. 22, for example, is a schematicdiagram illustrating a grid element 2200 of a polymorphic surface inwhich a display tile 2202 is integrated. As discussed above, the displaylayer 104 may be projected from above onto the display tile 2202 (whichmay comprise, for example, and LCD or rear projection film). Thus, themulti-layer “sandwich” embodiment illustrated in FIG. 1 is shown purelyfor illustrative purposes. As a further example, the observation layer106 need not consist of a single physical layer, and instead may includesensors that are physically positioned in various locations relative tothe display layer 104 and the polymorphic layer 102.

Thus, the various layers of the device 100 cooperate to provide avariety of user interfaces, responsive to dynamically changing usecontexts. For instance, the device 100 can simulate an alphanumerickeyboard (whose key position may or may not be adjustable), a computermouse, a track ball, a track pad, a scratch pad, a track point, abass-relief sculpting toy (e.g., using the display to visually emphasizeshapes that are “carved” by the user's fingers), a potter's wheel (e.g.,using a tactile “line” to act as a lathe upon a rotationally symmetricsolid), a musical instrument (e.g., using a tactile set of “strings”that move past the user's fingers under pressure), a set of fingerpaints, or any of the other previously described user input devices.

FIG. 5 is a schematic diagram illustrating a top view of one embodimentof the polymorphic layer 102, according to the present invention. Asdiscussed above, in an illustrative embodiment, the polymorphic layer102 may be formed of a plastic (e.g., acrylic) three-dimensionaltransparent micro grid 500 comprising a plurality of grid elements andat least one actuator. As discussed above, the polymorphic layer 102 maybe configured in a variety of ways and may be formed from a variety ofmaterials. Thus, FIG. 5 illustrates only one way in which thepolymorphic layer 102 may be configured. In addition, the role of theactuator may be performed by human activity or another environmentalforce which acts upon the grid elements. For example, a person can pressdown on the grid elements or rotate the display so that gravity acts onthe grid elements.

In one embodiment, the grid elements are arranged in a plurality ofintersecting rows 502 ₁-502 _(n) (hereinafter collectively referred toas “rows 502”) and columns 504 ₁-504 _(m) (hereinafter collectivelyreferred to as “columns 504”). Thus, a grid element is positioned ateach intersection of a row 502 and column 504. For ease of illustration,only a single grid element 506 is numbered in FIG. 5, at theintersection of row 502 ₁ and column 504 ₁. Grid elements are generallyreferred to as “grid elements 506” hereinafter.

At least one actuator 508 is coupled to the grid 500. In one embodiment,the actuator 508 is a substantially global actuator. That is, theactuator 506 is capable of driving a plurality of the grid elements 506;thus, an individual actuator is not needed to drive each grid element506. In one embodiment the actuator 508 drives substantially all of thegrid elements 506. In another embodiment, a plurality of actuators 508is deployed, such that each actuator 508 drives a particular localizedregion or group of grid elements 506. In yet another embodiment, aplurality of actuators 508 is deployed, such that each actuator 508drives at least one row 502 or at least one column 504. For example,each row 502 or each column 504 may be driven by a dedicated actuator508.

Each grid element 506 is capable of being displaced or elevated tomultiple heights by its associated actuator 508, for example using airpressure or electromechanical actuation (such as a solenoid). Thus, theactuator 508 drives the grid elements 506 in a direction that issubstantially normal to the interaction surface of the polymorphic layer102.

One embodiment of a polymorphic layer 102 additionally includes at leastone selective or substantially local clamping mechanism. A selectiveclamping mechanism controls the displacement of a specific grid element506 (or group of grid elements 506) by the actuator 508. FIGS. 6-10illustrate embodiments of various clamping mechanisms that may be used.For simplicity's sake, the actuator or reset mechanism is notillustrated in these Figures.

FIG. 6, for example, is a schematic diagram illustrating across-sectional view of one embodiment of the polymorphic layer 102illustrated in FIG. 1. As discussed above, the polymorphic layer 102 maybe configured in a variety of ways and may be formed from a variety ofmaterials. Thus, FIG. 6 illustrates only one way in which thepolymorphic layer 102 may be configured.

Generally, the polymorphic layer 102 illustrated in FIG. 6 useselectrostatic clamping to control the displacement of individual gridelements 506. This allows greater control over smaller regions of thepolymorphic layer 102 while requiring little power to operate. Moreover,this approach is compatible with both force-based touch screen andfrustrated total internal reflection (FTIR) devices.

As illustrated, the polymorphic layer 102 generally comprises aplurality of posts 600 ₁-600 _(n) (hereinafter collectively referred toas “posts 600”), where each post is positioned beneath a grid element506 or group of grid elements 506. The displacement of each of the posts600 (and corresponding grid elements 506) is controllable using anelectrostatic selector mechanism 614 ₁-614 _(n) (hereinaftercollectively referred to as “selector mechanisms 614”). The posts 600and selector mechanisms 614 are positioned within several layers ofmaterial, including: a post grid 602, a standoff grid 604, and agel/elastomer layer 606. Optionally, the layers of material mayadditionally include one of more of: a top membrane 608, a top grid 610,and a resistive multi-touch sensor layer 612. The spaces between thevarious layers may be filled with a gel or liquid having a refractiveindex that matches a refractive index of the various layers. In analternative embodiment, all of the layers are themselves compliant orflexible materials so that the resulting structure is also compliant andflexible (and all elements may have closely matching indices ofrefraction).

Each post 600 (and, by association, each grid element 506) can bedisplaced by an amount that is selected from among a plurality ofnon-zero amounts. FIG. 6 in particular illustrates three differentdisplacements or positions: neutral (i.e., not activated, as illustratedby the post 600 _(n)), raised (i.e., activated, as illustrated by theposts 600 ₂ and 600 ₃), and selected (i.e., raised and subsequentlypressed by a user, as illustrated by the post 600 _(n)). Each of theposts 600 is moveable to one of these positions by activating theactuator 508 illustrated in FIG. 5 to drive the post 600, and thenactivating an associated selector mechanism 614 that is coupled to thepost 600 to control the amount by which the post 600 is driven. Theselector mechanisms 614 are spring loaded to allow vertical movement ofthe posts 600.

The selector mechanisms 614 may take a variety of forms; two of theseforms are illustrated in FIG. 6. In one embodiment, the selectormechanism 614 is a collapsible mechanism that expands when theassociated post 600 is raised and contracts when the post 600 is loweredor when the post 600 is raised and subsequently selected (e.g., by theuser pressing down on the post 600). The collapsible mechanism mayinclude for example, an elastomeric carrier membrane positioned betweena pair of living hinges. The elastomeric carrier membrane provides aspring load. This embodiment is illustrated by the selector mechanisms614 ₁, 614 ₂, and 614 _(n).

In another embodiment, the selector mechanism 614 is a trap doormechanism comprising a counter post that biases two overlapping,electrostatically clamped leaves. The leaves rise with the post 600 andlower or flatten when the post 600 is lowered or when the post 600 israised and subsequently selected. This embodiment is illustrated by theselector mechanism 614 ₃ and relies on the compliance of thegel/elastomer layer 606 below. This embodiment is illustrated in moredetail in FIGS. 7A-7C, which illustrate various embodiments of a trapdoor-like selector mechanism.

FIG. 7A, for example, illustrates a selector mechanism 614 comprisingtwo overlapping, electrostatically clamped leaves 700 ₁ and 700 ₂(hereinafter collectively referred to as “leaves 700”). The leaves 700may comprise, for example, a dielectric material layered over atransparent electrode (e.g., an indium tin oxide or electrostaticdischarge polymer electrode). An example of a dielectric material thatmay be suitable for electrostatic clamping is polyurethane, such asDeerfield PT 6100S polyurethane. An example of a transparentelectrostatic discharge (antistatic) polymer is AM Corp. 3M 40antistatic tape. Electrostatic clamping is achieved when the leavestouch each other and a voltage differential exists between theelectrodes on each leaf. The dielectric material on either or bothleaves maintains the voltage differential. Electrostatic clampingtechnology is familiar to those of skill on the art (See, e.g., U.S.Pat. No. 7,598,651).

In another embodiment illustrated in FIG. 7B, the leaves 700 areflexible, so that they can better conform to each other and better lockunder pressure. In the illustrated embodiment, the leaves 700 form anarch that demonstrates bistable ability similar to a passive domeswitch. However, flexible leaves 700 may also be utilized in a mannerthat does not form an arch.

In another embodiment illustrated in FIG. 7C, the leaves 700 lock in a“downward” position when the electrostatic clamping is applied after theassociated post 600 is pressed down by a user, eliminating the need forthe latching to counter the force of the press. The post 600 can bepressed further downward to activate a switch, enable FTIR detection, orother touch-based or proximity-based (e.g., using capacitance to measurea conductive surface in close proximity to another surface) detection.

FIG. 7D illustrates an alternative selector mechanism. In thisembodiment, the leaves 700 clamp directly to the associated post 600.The flexibility of the leaves 700 allows the post 600 to be presseddownward (and to exhibit some non-linear spring forces similar to a domeswitch). The post 600 can be pressed further downward to activate aswitch, enable FTIR detection, or other touch-based or proximity-baseddetection.

“Self-locking” designs for clamps, brakes, and similar mechanisms arefamiliar to practitioners of skill in the art. The above clampingmechanisms can benefit from self-locking designs, in some embodiments,such that the force of a person pressing acts to further press togetherand confirm the leaves 700. In this way, a post can be further depressedwithout causing the clamping to slip.

FIG. 23 is a schematic diagram illustrating one embodiment of a gridelement array 2300 incorporating selector mechanisms such as thoseillustrated in FIG. 7C. In particular, the leaves 2302 are affixed to agrid plate 2304, which provides electrical connections. The grid plate2304 includes a plurality of apertures, each aperture housing a pair ofleaves 2302. For ease of illustration, only single pair or leaves 2302(and a single electrical connection 2306) is illustrated. Multiple gridplates such as the grid plate 2304 may be stacked and aligned with thevarious layers of the adaptable I/O device 100

FIG. 8 illustrates a close-up view of the selector mechanism 614illustrated in FIG. 6. As illustrated in FIG. 8, the posts 600 areremoved completely, and the selector mechanisms 614 in essence becomeboth the posts and the selector mechanism.

FIGS. 9A-9B illustrate another alternative embodiment of the selectormechanism 614 illustrated in FIG. 6. In particular, FIGS. 9A-9Billustrate a selector mechanism 900 that is configured as a dome switch.In FIG. 9A, the dome switch comprises a transparent plastic dome switchhaving a plurality of wings 902 ₁-902 _(n) (hereinafter collectivelyreferred to as “wings 902”). When the dome switch is depressed, thewings 902 move outward as illustrated by the arrows. The wings 902 mayalso be clamped to an underlying layer 906 so that the switch requiresgreater pressure to depress. As with the other selector mechanismsillustrated in FIGS. 6 and 8, if the pressure is applied by a globalactuator to depress all of the switches, then a clamped dome switchwould tend to extend higher than the unclamped switches.

In FIG. 9B, the wings 902 include overlapping sliding electrolaminatescales 904 ₁-904 _(n) (hereinafter collectively referred to as “scales904”). The scales 904 allow the dome switch to be locked in a depressedposition. As described above with reference to FIG. 2, the adjustment ofthe clamping of these dome switches or other selector mechanisms mayalso be done while the user is pressing the button, post, or switch. Inthis manner, the feel of the switch can be made to simulate a variety ofresponses. For example, the familiar “click” feel of a key on aconventional keyboard can be simulated by releasing the clamping as theswitch is depressed.

Further embodiments of the polymorphic layer 102 use different types ofclamping mechanisms, including other electrically controllable clampingmethods such as electrochemically or electrothermally controlledclamping mechanisms, electroactive polymer mechanisms, electromagneticclamping mechanisms (e.g., using magnetic latching), mechanical clampingmechanisms (e.g., mechanical levers, strings, straps, locking pins, etc.driven by actuators such as electroactive polymers or electromagneticdevices such as solenoids), and ferrofluids (also referred to as“magnetorheological fluids”) or electrorheological fluids.

As discussed above, the chosen clamping mechanism restricts the movement(e.g., vertical displacement) of at least some of the grid elements 506.That is, the clamping mechanism partially counteracts the actuator 508by controlling the amount by which an associated grid element 506 isdisplaced by the actuator 508. For example, the actuator 508 and theclamping mechanism may cooperate to ensure that only a selected set ofgrid elements 506 is elevated at a given time. For instance, activationof the actuator 508 may cause all grid elements 506 in a given row 502to be elevated, while activation of the clamping mechanism may cause allgrid elements 506 in a column 504 intersecting the given row 502 to beheld in a non-elevated position.

FIG. 10 is a schematic diagram illustrating a cross-sectional view ofanother embodiment of the polymorphic layer 102 illustrated in FIG. 1.As illustrated, the polymorphic layer 102 generally comprises aplurality of posts of buttons 1000 ₁-1000 _(n) (hereinafter collectivelyreferred to as “buttons 1000”), where each button 1000 is positionedbeneath a grid element 506 or group of grid elements 506. The buttons1000 are clamped in place associated latches 1002 ₁-1002 _(n)(hereinafter collectively referred to as “latches 1002”), where eachlatch 1002 comprises a pair of leaves that clamp together.

The upper leaf of each latch 1002 clamps to a first clamping layer 1004,while the lower leaf of each latch 1002 clamps to a second clampinglayer 1006 located below the first clamping layer 1004. Additionally,the lower leaf clamps to a pull layer 1008 located below the secondclamping layer 1006. The pull layer 1006 is movable in both thehorizontal and vertical directions.

The displacement of each of the buttons 1000 (and corresponding gridelements 506) is controllable using the latches 1002. As describedabove, the buttons 1000 may be held in a lowered position by clampingthe leaves of the associated latches 1002 together. The inclusion of thesecond clamping layer 1006, which is positioned between the latches 1002and the pull layer 1006, allows one to control which buttons 1000 arepulled into the lowered position and by what amount the buttons 1000 arelowered. Greater variations in the displacement of the buttons 1000 canbe achieved using “inchworming” (i.e., repeated back and forth,horizontal motion of the pull layer 1008), which alternates the clampingbetween the leaves of the latches 1002 that act between the first andsecond clamping layers 1004 and 1006 and the lower leaf clamps acting onthe pull layer (thereby pulling down or pushing up on the upper leaves).The pull layer 1008 may be moved by the global actuator 508. In thiscase, the global actuator 508 can move back and forth in smallincrements and enable the use of additional small-amplitude actuationtechnologies such as piezoelectrics or microelectromechanical systems(MEMS) electrostatic actuators.

The array of selectively clamped grid elements can be arrayed on asubstantially flat surface, but can also be arrayed on curved (or othernon-flat) surfaces. For example, FIG. 21 is an exploded viewillustrating a portion of adaptable input/output device 2100 having acurved surface. The curved surface may be advantageously used to producea device in which the polymorphic display dynamically reproduces theshape and feel of three-dimensional curved objects, such as human headsand faces, planetary globes, and pottery vessels. This allows theadaptable input/output device 2100 to take the shape of a human head,human hand, or human face. Thus, conversing users may “shake hands” orview a physical representation of each other through the adaptableinput/output device 2100. In one embodiment, for instance, the surfaceof the adaptable input/output device 2100 roughly takes the shape of ahuman head or face in its un-deformed form, and then adopts moreuser-specific features (e.g., size, shape, and placement of the nose,eyes, etc.) when deformed. The surface of the adaptable input/outputdevice 2100 could adapt continuously such that it is “animated” with therepresented user's real-time movements, which may directly sensed orindirectly inferred (e.g., by observation using one or more videocameras or by synchronizing mouth movements with detected speech). Asone approach, pneumatic or hydraulic actuators may be used in suchembodiments, in which a gas or fluid is pumped into an elastic bladder2102 that is located within the grid plate 2104. Inflation and deflationof this bladder 2102 provide actuation pressure that can be used toraise, lower, or provide the desired reaction force to each grid elementbutton 2106.

As discussed above, the clamping mechanisms deployed in the polymorphiclayer may also comprise lockable joints, rather than movable lockingposts or pins. FIGS. 11A-11C, for instance, are schematic diagramsillustrating top views of various exemplary embodiments of clampingmechanisms employing lockable joints, according to the presentinvention.

FIG. 11A, for example, illustrates a mesh 1100 a of connectors or rigidbars 1102 a, where the rigid bars 1102 a meet each other at pivotingjoints 1104 a. Each rigid bar 1102 a is coupled to at least two joints1104 a. For ease of illustration, only one rigid bar 1102 a and twojoints 1104 a are indicated by reference numerals in FIG. 11A. Portionsof the mesh 1100 a may be locally raised and lowered by bending(rotation between the rigid bars 1102 a) at the joints 1104 a. AlthoughFIG. 11A depicts a mesh whose apertures are substantially rectangular inshape and rigid bars 1102 a that are substantially cross-shaped, othermesh and bar shapes are possible, as illustrated in FIGS. 11B-11C.

FIG. 11B, for example, illustrates a mesh 1100 b whose apertures aresubstantially triangular in shape, formed of rigid bars 1102 b that aresubstantially star-shaped. Each rigid bar 1102 b is coupled to at leastthree joints 1104 b.

FIG. 11C illustrates a mesh 1100 c whose apertures are substantiallyhexagonal shape, formed of rigid bars 1102 c that are substantiallyY-shaped. Each rigid bar 1102 c is coupled to at least two joints 1104c.

Although not illustrated, additional shapes for the mesh apertures andthe rigid bars may be deployed without departing from the scope of thepresent invention.

Although the rigid bars 1102 a-c illustrated in FIGS. 11A-11C aredescribed as “rigid,” in certain embodiments, the rigid bars 1102 a-cmay be formed of a material that offers a degree of flexibility.Alternatively, the degrees of freedom allowed by the lockable joints1104 a-1104 c can be varied by varying the standoff distances in thejoints 1104 a-1104 c or by maintaining a portion of the flexible orcompliant fabric disposed above the meshes 1100 a-1100 c in a permanent,partially folded state.

In one embodiment, the joints 1104 a-1104 c include integral sensorsthat detect forced bending of the joints 1104 a-1104 c or stress.

FIGS. 12A-12C are schematic diagrams illustrating various embodiments oflocking mechanisms that may be employed to selectively lock the lockablejoints 1104 a-c illustrated in FIGS. 11A-11C. FIG. 12A, for instance,illustrates a fork-like locking mechanism 1200 a; FIG. 12B illustratesan “earmuff”-like locking mechanism 1200 b; and FIG. C illustrates adirect bar-to-bar locking mechanism 1200 c. Any of the lockingmechanisms 1200 a-1200 c may be strengthened using retaining pins (notshown) to secure the rigid bars 1102 to the locking mechanisms 1200a-1200 c. However, it is important that the joints allow for not justbending, but also some lateral motion (i.e., the bars 1102 a should beable to move closer and farther apart). This motion allows the bendingof the joints to achieve the desired arbitrary surface shape. Locking atthe joints may be achieved, for example, by clamping the bars 1102 toeach other or to the locking plates 1200. Clamping may be byelectrostatic attraction or by any of the other means described above.

As discussed above, the polymorphic layer 102 may also comprise aplurality of layers of a flexible corrugated or laminar plastic orcompliant fabric. FIG. 13, for example, illustrates one embodiment of apiece 1300 of flexible or compliant fabric. As illustrated by the arrows1302 and 1304, the fabric stretches more in one direction (e.g., the xdirection) than it does in another direction (e.g., the y direction). Aplurality of layers of such a fabric may be arranged and selectivelyclamped together in selected regions. By clamping two orthogonallycompliant fabric layers together, the resulting fabric stack is made tobe more rigid (non-stretchable) in the area where clamping is enabled.In another embodiment, at least three layers of plastic or fabric arearranged at substantially triangular or hexagonal positions relative toeach other. In further embodiments, additional layers of fabric andarrangements are possible.

In one embodiment, the fabric takes on a corrugated or accordion foldedshape in the direction in which it stretches more. That is, the fabrictakes on the accordion folded shape when at rest. When force is exertedon one or more ends of the fabric, the accordion folded shape isflattened, and the fabric stretches. FIG. 14, for example, illustratesone embodiment, of an accordion folded fabric 1400 that may be deployedabove any of the clamping mechanisms discussed herein. As illustrated,the accordion folded fabric 1400 is supported by one or more elasticbands 1402 a-1402 b that pull the accordion folded fabric 1400 in adirection that relaxes the accordion folds.

In yet another embodiment, multiple layers of fabric may be arranged toform addressable clamping regions. FIG. 15, for example, illustratesanother embodiment of a multi-layered fabric that may be deployed aboveany of the clamping mechanisms discussed herein. As illustrated, a firstlayer 1500 of fabric has a plurality of addressable clamping regions1502 ₁-1502 _(n) (hereinafter collectively referred to as “addressableclamping regions 1502”) formed thereon. A similarly-formed second layerof fabric (not shown) may be layered on top of the first layer 1500, butrotated approximately ninety degrees (such that the first layer 1500stretches more in a first direction, while the second layer stretchesmore in a second direction that is substantially orthogonal to the firstdirection). A third layer of fabric may be positioned between the firstlayer 1500 and the second layer. To achieve electrostatic clamping, thethird layer of fabric is then raised to a voltage potential compared tothe first layer 1500 and the second layer. Thus, the clamping occursbetween third (middle) layer and the first layer 1500, and between thethird layer and the second layer. Thus, the third layer effectivelyforms a tension wire or non-extensible fabric. In order to maintain thisvoltage potential, the fabric layers must include a conductive electrodelayer. Any one of two adjacent fabric layers must also include adielectric layer that insulates the adjacent electrodes from each other.

Alternatively, the addressable clamping regions 1502 may be formed ofaddressable, chargeable conductors that are “sewn” into the first layer1500 (where at least the second layer is formed similarly). Thechargeable conductors may be, for example, electrically chargeable byapplying a voltage differential between two conductors. The layers canthen be clamped in discrete regions whenever a positively chargedaddressable clamping region 1502 on the first layer is positioned abovea negatively charged addressable clamping region on the second layer. Ifthe portions of the first layer 1500 that do not comprise theaddressable clamping regions 1502 are capable of significant stretching,then, since slippage can occur between the layers, the unclampedassembly of layers will be very flexible; alternatively, when clamped,the assembly of layers will be very rigid.

FIG. 16 illustrates another embodiment of a multi-layered fabric thatmay be deployed above any of the clamping mechanisms discussed herein.Specifically, FIG. 16 illustrates an array 1600 of interwoven ribbons1602. For ease of illustration, only one of the ribbons 1602 isindicated by a reference numeral in FIG. 16. The ribbons 1602 are freeto slide relative to each other in the unclamped state. The ribbons 1602may be clamped together at the locations at which they cross and therebymake the resulting structure rigid in the vicinity of the clamping. Inone embodiment, the ribbons 1602 are substantially flat. In oneembodiment, the ribbons 1602 are formed of a meta-material, for exampleas described in U.S. Pat. No. 7,598,651, such that the ribbons 1602 maybe stiff or floppy. Although FIG. 16 illustrates an array 1600 in whichthe ribbons 1602 are woven in two directions, the ribbons may be wovenin additional directions (e.g., three or more) to increase strength.

In addition to the embodiments illustrated in FIGS. 11-16, otherembodiments of shape lockable surfaces are possible. Theabove-referenced U.S. Pat. No. 7,598,651, for example, describes severalsuch embodiments. The embodiments described herein are merelyillustrative of how shape lockable surfaces in general may beincorporated into a polymorphic display structure.

FIG. 17 is a flow diagram illustrating one embodiment of a method 1700for interacting with a user of a computing system, according to thepresent invention. The method 1700 may be implemented, for example, byany embodiment of the hardware device illustrated in FIGS. 1-16. Assuch, reference is made in the discussion of the method 1700 to variouselements of the device 100. It will be appreciated, however, that themethod 1700 is not limited to implementation with a device configuredexactly as illustrated in FIG. 1. That is, the method 1700 may beimplemented in hardware devices having configurations that differ fromthat illustrated in FIG. 1.

The method 1700 is initialized in step 1702 and proceeds to step 1704,where the observation layer 106 monitors the user's actions and otherobjects or actions at a distance from the display, preferably includingthe objects' or actions' proximity to the display surface. In oneembodiment, the user actions that are monitored include the user's handpositions and gestures (which may include gestures other than handgestures). In further embodiments, the user's gaze may be tracked bysensors, such as by tracking the direction of the user's nose and/or byfollowing the user's eyes. These actions may be directly monitored bythe sensors embedded in the observation layer 106. In one embodiment,relevant computing device display parameters that are taken into accountinclude the current active applications, the currently active inputfields associated with the currently active applications, and thedisplay configuration (e.g., single monitor versus multiple monitors).These parameters may be directly monitored by the sensors embedded inthe observation layer 106 and/or may be transmitted directly from thecomputing system 108.

In step 1706, the observation layer 106 is utilized to infer a currentuse context from the monitored information. For example, if a currentlyactive input field visible of the computing device display includes aplurality of free form fields, and if the observation layer 106 detectsthat the user's hands are currently positioned as if to type, then thesystem may infer that the current use context involves the user typingsome sort of free form text. The ability to make such inferences may belearned over time as the hardware device 100 adapts to the user.Additionally, the use context may be determined by the computing system108 based on other factors, including the state of applicationscurrently being executed and interacted with by the user.

In step 1708, the polymorphic layer 102 and the display layer 104 adjustin response to the inferred use context. For example, continuing theexample above, if the current use context involves the user typing somesort of free form text, the polymorphic layer 102 may dynamically adjustits configuration (e.g., by adjusting the configuration of the gridelements, joints, and/or compliant fabric) such that a portion of thedisplay layer 104 takes the shape of a standard keyboard, preferably ata convenient location based upon factors such as the positioning of theuser's hands and/or the focus of the user's gaze. Additionally, in someembodiments, the keyboard may be further configured for the user'schosen language, layout, design, ergonomic factors, colors, sensitivity,and the like. Thus, a deformation is created in the interaction surface.For example, continuing the example above, if the current use contextinvolves the user typing some sort of free form text, the polymorphiclayer 102 might be configured as a set of alphanumeric keyboard keys.

In an alternative embodiment of steps 1706-1708, the user interactivelyrequests or selects a desired user interface device, and the polymorphiclayer 102 adapts in response to provide the desired user interfacedevice. For example, the user might gesture with typing hands to requesta keyboard. In this case, the observation layer 106 would receive thegesture, and in response the polymorphic layer 102 would deform toprovide a keyboard. Alternatively, the user might gesture or otherwiseenter a command to provide a user interface device menu (i.e., aninteractive menu from which the user can select from choices like“keyboard,” “joystick,” and the like). The polymorphic layer 102 willthen deform to provide the selected user interface device.

In optional step 1710 (illustrated in phantom), the observation layer106 calculates the optical effects of the adjusted polymorphic layerconfiguration. In some cases, adjustment of the polymorphic layer 102 instep 1708 may optically distort the appearance of the underlying displaylayer 104. In such cases, it may be beneficial to compensate for thesedistortions so that the display layer 104 appears as intended. In oneembodiment, the optical effects are calculated accounting for theinferred or measured position of the user's eyes relative to the displaylayer 104, and in step 1712, the display is modified accordingly toproduce the desired effect by the viewer. This effectively reverses someor all of the optical distortion that is introduced by the initialadjustment of the polymorphic layer 102 in step 1708. In alternativeembodiments, as previously discussed, where the display surface is notpositioned beneath the polymorphic layer 102, optical distortion of thisnature is not an issue.

In step 1714, the polymorphic layer 102 receives tactile input from theuser. For example, continuing the example above, the tactile input mayinclude the press of several buttons on a conventional keyboardconfiguration to spell out one or more words. In another embodiment, thetactile input may include the molding of the polymorphic layer 102 intoa three-dimensional shape. In step 1716, the observation layer 106transmits the input (e.g., the one or more words) to the computingsystem 108 for further processing. In one embodiment, the transmissionof the input may also involve making corrections to the input (e.g.,over time, the hardware device 100 may learn common input errors thatthe user tends to make). The method 1700 then returns to step 1704 andcontinues to monitor the user's actions and the display parameters sothat the hardware device 100 continuously and dynamically adapts tochanging use contexts.

Thus, the user does not have to change the position of his fingers onthe hardware device 100. Instead, the hardware device 100 detects thelocations of the user's fingers and responsively positions theappropriate user interfaces and interaction models.

FIG. 18 is a flow diagram illustrating one embodiment of a method 1800for adjusting the polymorphic layer 102 of the hardware device 100.Thus, the method 1800 may be implemented in accordance with step 1710 ofthe method 1700. Although reference is made to various elements of FIGS.1-16, it will be appreciated that the method 1800 is not limited toimplementation with a device configured exactly as illustrated in FIGS.1-16.

The method 1800 is initialized at step 1802 and proceeds to step 1804,where the polymorphic layer 102 receives a signal indicating that theconfiguration of the polymorphic layer 102 should be adjusted. Forexample, the signal might indicate that the polymorphic layer 102 shouldbe configured as an alphanumeric keyboard. In one embodiment, the signalis received from the computing system to which the hardware device 100is coupled.

In step 1806, the clamping mechanism (e.g., electrostatic latches,locking pins, lockable joints, layered fabrics, or any of the otherembodiments described above) selectively locks one or more local regionsof the polymorphic layer's interaction surface. In particular, theportions of the clamping mechanism that control the one or more regionsof the interaction surface are locked. In one embodiment, these regionsof the interaction surface are locked in a downward (not raised)position. In another embodiment, these regions of the interactionsurface are locked in a raised position. The specific regions of theinteraction surface that are locked, as well as the position in whichthe regions are locked, will depend on the current configuration of thepolymorphic layer 102 and the desired configuration of the polymorphiclayer 102 as indicated by the signal received in step 1804. Thus, thelocked and un-locked regions of the interaction surface are dynamicallydefined responsive to the received signals.

In step 1808, the global pressure is increased (e.g., by activating theactuator). This will cause upward or outward motion of any regions ofthe interaction surface that have not been selectively locked in step1806. The result is an interaction surface having the three-dimensionalshape and feel of a desired input device (or other interactive shape).If more than one gradation in upward or downward motion is desired, thenthe actuation can be varied to change the global pressure in concertwith selective locking. In this case, a region that is locked only whenthe upward pressure achieves a certain level would have a greater motionthan a region that is locked at a lower pressure, for example.

The method 1800 then returns to step 1804 and awaits a next signal toadjust the configuration of the polymorphic layer 102.

In some embodiments, the hardware device 100 may be specifically trainedfor cognitive and motion models associated with neurological and nervoussystem disorders such as Parkinson's disease, multiple sclerosis,Alzheimer's disease, and the like. This will enable dynamic correctionof inputs resulting from jittery movements and support easiercross-application automation.

Further extensions of the hardware device 100 include use with dualscreen displays and dual graphics processing units (GPUs). For example,one GPU may be used to accelerate the graphics output of the other GPUor to accelerate the streaming cores to real time process gestures andinteractions.

Still further extensions of the hardware device 100 allow any interfaceto be changed into another. This capability may prove useful in combatsituations or in driving emergencies, among other scenarios. Forinstance, rather than bring several different devices into suchscenarios, it may only be necessary to bring one device (e.g., thehardware device 100) that can transform into several different devices.For example, a single device could transform from an alphanumeric keypadfor a cellular telephone to a global positioning system (GPS) unitinterface to a controller for a small robot. The device could transformbased on its proximity to certain objects. For example, an adaptivedevice in accordance with the present invention could be used tointeract with a bank automatic teller machine (ATM). In furtherembodiments, the hardware device 100 could be used for musical orartistic instruction (e.g., where the hardware device 100 transformsinto an interface that simulates a piano, a set of drums, a fingerpainting surface, a potter's wheel, or the like).

In further extensions, the hardware device 100 is integrated in acellular telephone. FIG. 19, for example, illustrates a cellulartelephone 1900 having an integrated hardware device such as the hardwaredevice 100 illustrated in FIG. 1. In the instance of FIG. 19, thepolymorphic layer of the hardware device may comprise, for example, alayer of material that can be electronically controlled in selectiveareas. For example, electrostatic clamping may be used to stiffenselected areas of the material, such as the area designated 1902 in FIG.19. Force may be applied beneath the layer of material (e.g., usingpumped fluid or gas, electronic drivers, or other actuation meansincluding those discussed above, such as with reference to FIG. 10)which results in the raising or lowering of tactile bumps 1904 ₁-1904_(n) in areas where the layer of material is not stiffened.Alternatively, the entire layer of material (or a majority of the layerof material) can be stiffened in a raised, lowered, or neutral state toproduce a single button or shape. Thus, the hardware device can betransformed into an interface that allows substantially any type ofcellular telephone interaction, including keyboard typing, scrolling,shrinking, or the like.

A hardware device integrated in a cellular telephone would allow a userto control the telephone simply by touch. Unlike conventional touchscreen interfaces, however, the hardware device additionally providestactile feedback (to the single fingertip level) that allows the user tocontrol the cellular telephone without having to constantly look at thetelephone's screen.

FIG. 20 is a high level block diagram of the present inventionimplemented using a general purpose computing device 2000. It should beunderstood that embodiments of the invention can be implemented as aphysical device or subsystem that is coupled to a processor through acommunication channel. Therefore, in one embodiment, a general purposecomputing device 2000 comprises a processor 2002, a memory 2004, aninput/output (I/O) adjustment module 2005, and various input/output(I/O) devices 2006 such as a display, a keyboard, a mouse, a modem, amicrophone, speakers, a touch screen, an adaptable I/O device, and thelike. In one embodiment, at least one I/O device is a storage device(e.g., a disk drive, an optical disk drive, a floppy disk drive).

Alternatively, embodiments of the present invention (e.g., I/Oadjustment module 2005) can be represented by one or more softwareapplications (or even a combination of software and hardware, e.g.,using Application Specific Integrated Circuits (ASIC)), where thesoftware is loaded from a storage medium (e.g., I/O devices 2006) andoperated by the processor 2002 in the memory 2004 of the general purposecomputing device 2000. Thus, in one embodiment, the I/O adjustmentmodule 2005 for adjusting an adaptable I/O device described herein withreference to the preceding Figures can be stored on a non-transitorycomputer readable medium (e.g., RAM, magnetic or optical drive ordiskette, and the like).

It should be noted that although not explicitly specified, one or moresteps of the methods described herein may include a storing, displayingand/or outputting step as required for a particular application. Inother words, any data, records, fields, and/or intermediate resultsdiscussed in the methods can be stored, displayed, and/or outputted toanother device as required for a particular application. Furthermore,steps or blocks in the accompanying Figures that recite a determiningoperation or involve a decision, do not necessarily require that bothbranches of the determining operation be practiced. In other words, oneof the branches of the determining operation can be deemed as anoptional step.

Although various embodiments which incorporate the teachings of thepresent invention have been shown and described in detail herein, thoseskilled in the art can readily devise many other varied embodiments thatstill incorporate these teachings.

What is claimed is:
 1. A hardware device for facilitating an interactionbetween a computing system and a user, the hardware device comprising:an interaction surface for supporting the interaction; a single actuatorcapable of driving a first region of the interaction surface; and afirst selective clamping mechanism capable of restricting movement ofone or more second regions of the interaction surface that are coplanarwith the first region and that partly intersect the first region, therestricting at least partially counteracting an effect of the actuatoron a portion of the one or more second regions, wherein a displacementof one or more desired portions of the interaction surface isdynamically controllable.
 2. The hardware device of claim 1, wherein theactuator is capable of driving the first region in a directionsubstantially normal to the interaction surface, and wherein thedisplacement is substantially normal to the interaction surface.
 3. Thehardware device of claim 1, wherein the displacement of the desiredportions equals one or more distances dynamically selected from among aplurality of non-zero distances.
 4. The hardware device of claim 1,wherein the displacement is reversible in response to a user-exertedforce.
 5. The hardware device of claim 4, wherein the user-exerted forceis a press, and the displacement is gradually reversible in response tothe press.
 6. The hardware device of claim 4, wherein a resistance ofthe one or more desired portions to the user-exerted force is variable.7. The hardware device of claim 1, further comprising: a controllableratcheting mechanism.
 8. The hardware device of claim 1, wherein thefirst region comprises substantially an entirety of the interactionsurface.
 9. The hardware device of claim 1, wherein each of the one ormore second regions is dynamically defined.
 10. The hardware device ofclaim 1, further comprising: a second selective clamping mechanismcapable of dynamically clamping together a plurality of selectedelements of the interaction surface so that the plurality of selectedelements moves as a group.
 11. The hardware device of claim 1, whereinthe interaction surface comprises: an array of elements organized into aplurality of rows and a plurality of columns, such that the plurality ofrows intersects the plurality of columns, wherein a first one of theactuator and the first selective clamping mechanism operates on aselected one or more of the plurality of rows, and a second one of theactuator and the first selective clamping mechanism operates on aselected one or more of the plurality of columns.
 12. The hardwaredevice of claim 1, wherein: the interaction surface comprises aplurality of pivoting joints and a plurality of connectors between theplurality of pivoting joints; and the first selective clamping mechanismis capable of restricting a movement of those of the plurality ofpivoting joints that are associated with the one or more second regions.13. The hardware device of claim 1, wherein: the interaction surfacecomprises a compliant layer; and the first selective clamping mechanismis capable of restricting a compliance of the compliant layer within theone or more second regions.
 14. The hardware device of claim 1, whereinthe first selective clamping mechanism comprises an electricallycontrollable clamping mechanism.
 15. The hardware device of claim 14,wherein the first selective clamping mechanism comprises electrostaticclamping.
 16. The hardware device of claim 1, wherein the firstselective clamping mechanism includes a clamping mechanism selected fromone or more of: electrochemically controlled clamping, electrothermallycontrolled clamping, electromagnetic clamping, mechanical clamping,ferrofluids, and electrorheological fluids.
 17. The hardware device ofclaim 1, wherein the first selective clamping mechanism comprises: aplurality of individually controllable latches corresponding to aplurality of individually controllable elements of the interactionsurface.
 18. The hardware device of claim 1, wherein the first selectiveclamping mechanism is substantially transparent.
 19. The hardware deviceof claim 1, wherein the interaction surface displays a visual output tothe user under a control of the computing system.
 20. The hardwaredevice of claim 1, wherein the actuator and the first selective clampingmechanism are controllable by the computing system, and the displacementis thereby dynamically controllable.
 21. The hardware device of claim 1,wherein an input signal is transmitted to the computing system inresponse to a physical manipulation by the user of the one or moredesired portions after a displacement of the one or more desiredportions is caused.
 22. The hardware device of claim 21, wherein thephysical manipulation includes one or more of: pressing or touching. 23.The hardware device of claim 22, wherein the one or more desiredportions function as one or more input devices selected from: analphanumeric keyboard, a telephone-style keypad, a numeric keypad, amedia player controller, a joystick, a video game controller, atelevision-style remote controller, a vehicle controller, or a robotcontroller.
 24. The hardware device of claim 22, wherein the one or moredesired portions function as an interactive command for a windowdisplayed on the interaction surface, the interactive command comprisingone or more of: close the window, minimize the window, or scroll acontent of the window.
 25. The hardware device of claim 22, wherein theone or more desired portions represent a topographic terrain map. 26.The hardware device of claim 21, wherein the physical manipulationcomprises molding the one or more desired portions into athree-dimensional shape.
 27. The hardware device of claim 1, wherein theinteraction surface has a substantially non-planar shape in an absenceof the displacement.
 28. The hardware device of claim 1, wherein a shapeof the displacement resembles a human head, a human hand, or a humanface.
 29. A method for facilitating an interaction between a computingsystem and a user via an interaction surface, the method comprising:driving a first region of the interaction surface, using a singleactuator; and restricting movement of one or more second regions of theinteraction surface that are coplanar with the first region and thatpartly intersect the first region, using a first selective clampingmechanism, the restricting at least partially counteracting an effect ofthe actuator on a portion of the one or more second regions, wherein adisplacement of one or more desired portions of the interaction surfaceis dynamically controllable.
 30. A computer readable storage devicecontaining an executable program for facilitating an interaction betweena computing system and a user via an interaction surface, where theprogram performs steps of: driving a first region of the interactionsurface, using a single actuator; and restricting movement of one ormore second regions of the interaction surface that are coplanar withthe first region and that partly intersect the first region, using afirst selective clamping mechanism, the restricting at least partiallycounteracting an effect of the actuator on a portion of the one or moresecond regions, wherein a displacement of one or more desired portionsof the interaction surface is dynamically controllable.
 31. A hardwaredevice for facilitating an interaction between a computing system and auser, the hardware device comprising: an adaptable surface forsupporting the interaction, the adaptable surface being dynamicallydeformable under a control of the computing system so as to produce adeformation in a given region of the adaptable surface; a controllableclamping mechanism capable of dynamically varying a resistance of thedeformation in the given region of the adaptable surface to movement inresponse to a pressure applied by the user; and circuitry fortransmitting an input signal to the computing system responsive to thepressure.
 32. The hardware device of claim 31, wherein the deformationsubstantially mimics an appearance and a pressure response associatedwith one or more input devices.
 33. The hardware device of claim 32,wherein the one or more input devices comprises at least one of: a key,a button, or a joystick.
 34. The hardware device of claim 31, whereinthe adaptable surface further comprises: a viewing surface that displaysvisual output under a control of the computing system.
 35. The hardwaredevice of claim 34, wherein at least the deformation comprises asubstantially transparent material.
 36. The hardware device of claim 34,wherein the adaptable surface comprises one or more display elementspositioned on top of the deformation.
 37. A method for facilitating aninteraction between a computing system and a user via an adaptablesurface, the method comprising: dynamically deforming the adaptablesurface under a control of the computing system so as to produce adeformation in a given region of the adaptable surface; dynamicallyvarying a resistance of the deformation in the given region of theadaptable surface to movement in response to a pressure applied by theuser, using a controllable clamping mechanism; and transmitting an inputsignal to the computing system responsive to the pressure.
 38. Themethod of claim 37, wherein the deforming is performed in response to arequest from the user.
 39. The method of claim 38, wherein the requestindicates a desired type of user interface device for use in providingthe input signal.
 40. A computer readable storage device containing anexecutable program for facilitating an interaction between a computingsystem and a user via an adaptable surface, where the program performssteps of: dynamically deforming the adaptable surface under a control ofthe computing system so as to produce a deformation in a given region ofthe adaptable surface; dynamically varying a resistance of thedeformation in the given region of the adaptable surface to movement inresponse to a pressure applied by the user, using a controllableclamping mechanism; and transmitting an input signal to the computingsystem responsive to the pressure.
 41. A hardware display device forfacilitating an interaction between a computing system and a user, thehardware display device comprising: an adaptable surface for supportingthe interaction, the adaptable surface being dynamically deformableunder a control of the computing system so as to produce a deformationthat substantially mimics an appearance associated with an input device,wherein the deformation is further deformable under a user-appliedpressure to assume a shape and a pressure response that substantiallymimics the input device when pressed; a viewing surface that displaysvisual output under a control of the computing system, wherein thevisual output is dynamically configurable to display a visual appearancethat substantially mimics the input device; and circuitry fortransmitting an input signal to the computing system in response to theuser-applied pressure.
 42. The hardware display device of claim 41,wherein the deformation substantially mimics an appearance associatedwith one or more input devices.
 43. The hardware display device of claim42, wherein the one or more input devices comprises at least one of: akey, a button, or a joystick.
 44. The hardware display device of claim41, wherein the adaptable surface further comprises: a viewing surfacethat displays visual output under a control of the computing system. 45.The hardware display device of claim 44, wherein at least thedeformation comprises a substantially transparent material.
 46. Thehardware display device of claim 43, wherein the adaptable surfacecomprises one or more display elements positioned on top of thedeformation.